3D video decoding apparatus and 3D video decoding method

ABSTRACT

A 3D decoding apparatus according to the present invention includes: a decoding unit which decodes left-eye and right-eye code signals to generate left-eye and right-eye decode signals; an error determining unit which determines an error of the left-eye and the right-eye code signals; an output determining unit which determines, when there is an error in one of the left-eye and the right-eye code signals, whether the one of the code signals that is determined as including an error has an error data mount equal to or greater than a first threshold; and an output unit which outputs neither the left-eye nor the right-eye code signal when the error data amount is smaller than the first threshold, and outputs only the decode signal obtained by decoding the other of the code signals when the error data amount is equal to or greater than the first threshold.

CROSS REFERENCE TO RELATED APPLICATION

This is a divisional application of application Ser. No. 13/189,834,filed on Jul. 25, 2011, now U.S. Pat. No. 8,577,208, which is acontinuation application of PCT application No. PCT/JP2010/005849 filedon Sep. 29, 2010, designating the United States of America, which claimsforeign priority of Japanese Patent Application No. 2009-258212 filed onNov. 11, 2009, the entire contents of each of which are incorporatedherein by reference.

BACKGROUND OF THE INVENTION

(1) Field of the Invention

The present invention relates to three-dimensional (3D) video decodingapparatuses and 3D video decoding methods and particularly to a 3D videodecoding apparatus which decodes the first code signal obtained bycoding a video signal of the first view, and the second code signalobtained by coding a video signal of the second view different from thefirst view.

(2) Description of the Related Art

There is a known 3D video display apparatus which displays 3D videoimages (multiview video images) that are two-dimensional (2D) videoimages which convey a stereoscopic perception to a viewer. For example,Patent literature 1: Japanese Unexamined Patent Application PublicationNo. 2001-186516 discloses a technique of coding and decoding such 3Dvideo images.

This 3D video display apparatus displays the images which convey astereoscopic perception to a viewer, by displaying a right-eye image anda left-eye image which have a parallax therebetween. For example, the 3Ddisplay apparatus displays the right-eye image and the left-eye imagealternately for each frame. In addition, the viewer uses a pair ofglasses which switch, for each frame, sights between the right eye andthe left eye. This allows the viewer to view the right-eye image withthe right eye only and the left-eye image with the left eye only and tothereby recognize, in three dimensions, the images which the 3D videodisplay apparatus displays.

SUMMARY OF THE INVENTION

However, in the case where there is data loss or corruption by an error,such 3D video images may be displayed as video which looks unnaturalbecause of an instantaneous large change in the depth-wise (i.e., in theprojection direction) display position in 3D presentation or because ofan instantaneous change to a 2D presentation.

Furthermore, also in the case of a trick playback such as thefast-forward playback, such unnatural video may be displayed due to theinstantaneous large change in the depth-wise display position in 3D orthe like causes.

Thus, an object of the present invention is to provide a 3D videodecoding apparatus and a 3D video decoding method, by which favorablevideo images can be generated in at least one of the cases with an errorand in the trick play mode.

In order to achieve the above object, a 3D video decoding apparatusaccording to an aspect of the present invention is a 3D video decodingapparatus which decodes a first code signal obtained by coding a videosignal of a first view, and a second code signal obtained by coding avideo signal of a second view that is different from the first view, the3D video decoding apparatus including: a decoding unit configured todecode the first code signal to generate a first decode signal, and todecode the second code signal to generate a second decode signal; anerror determining unit configured to determine, for each predetermineddata amount, whether or not there is an error in the first code signaland in the second code signal; an output determining unit configured todetermine, when the error determining unit determines that there is anerror in one of the first and the second code signals assigned withcorresponding presentation time points and that there is no error in theother of the first and the second code signals, whether or not the oneof the first and the second code signals that is determined as includingan error has an error data amount equal to or greater than a firstthreshold; and an output unit configured not to output the first or thesecond decode signal that corresponds to the one or the other of thefirst and the second code signals, when the output determining unitdetermines that the error data amount is smaller than the firstthreshold, and to output only the first or the second decode signalwhich is obtained by decoding the other of the first and the second codesignals, when the output determining unit determines that the error dataamount is equal to or greater than the first threshold.

Thus, the present invention is capable of providing a 3D video decodingapparatus and a 3D video decoding method, by which favorable videoimages can be generated in at least one of the cases with an error andin the trick play mode.

Further Information About Technical Background to This Application

The disclosure of Japanese Patent Application No. 2009-258212 filed onNov. 11, 2009 including specification, drawings and claims isincorporated herein by reference in its entirety.

The disclosure of PCT application No. PCT/W2010/005849 filed on Sep. 29,2010, including specification, drawings and claims is incorporatedherein by reference in its entirety.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects, advantages and features of the invention willbecome apparent from the following description thereof taken inconjunction with the accompanying drawings that illustrate a specificembodiment of the invention. In the Drawings:

FIG. 1 is a block diagram showing a structure of a 3D video displaysystem according to the first embodiment of the present invention;

FIG. 2 shows an example of 3D video signals according to the firstembodiment of the present invention;

FIG. 3 shows an example of a left-eye image and a right-eye imageaccording to the first embodiment of the present invention;

FIG. 4 shows another example of the 3D video signals according to thefirst embodiment of the present invention;

FIG. 5 is a block diagram showing a structure of a 3D video decodingapparatus according to the first embodiment of the present invention;

FIG. 6 shows a structure of an input video signal according to the firstembodiment of the present invention;

FIG. 7 shows a structure of a left-eye code signal according to thefirst embodiment of the present invention;

FIG. 8 shows a picture reference relation according to the firstembodiment of the present invention;

FIG. 9 is a flowchart showing a decoding process of the 3D videodecoding apparatus according to the first embodiment of the presentinvention;

FIG. 10 shows input video signals and output video signals in the casewhere there is an error in right-eye code signals, in the 3D videodecoding apparatus according to the first embodiment of the presentinvention;

FIG. 11 shows input video signals and output video signals in the casewhere there is an error in right-eye code signals, in the 3D videodecoding apparatus according to the first embodiment of the presentinvention;

FIG. 12 is a flowchart showing a decoding process of a 3D video decodingapparatus according to the second embodiment of the present invention;

FIG. 13 shows input video signals and output video signals in the casewhere there is an error in a non-reference coded picture, in the 3Dvideo decoding apparatus according to the second embodiment of thepresent invention;

FIG. 14 shows the input video signals and the output video signals inthe case where there is an error in a reference coded picture ofright-eye code signals, in the 3D video decoding apparatus according tothe second embodiment of the present invention;

FIG. 15 shows a relation between an error slice and a reference areaaccording to the second embodiment of the present invention;

FIG. 16 shows the input video signals and the output video signals inthe case where the error slice is not included in the reference area, inthe 3D video decoding apparatus according to the second embodiment ofthe present invention;

FIG. 17 shows the input video signals and the output video signals inthe case where the error slice is included in the reference area, in the3D video decoding apparatus according to the second embodiment of thepresent invention;

FIG. 18 is a block diagram showing a structure of a 3D video decodingapparatus according to the third embodiment of the present invention;

FIG. 19 shows a complementing process of the 3D video decoding apparatusaccording to the third embodiment of the present invention;

FIG. 20 is a flowchart showing the complementing process of the 3D videodecoding apparatus according to the third embodiment of the presentinvention;

FIG. 21 is a block diagram showing a structure of a 3D video decodingapparatus according to the fourth embodiment of the present invention;and

FIG. 22 is a flowchart showing a decoding process of the 3D videodecoding apparatus according to the fourth embodiment of the presentinvention.

DESCRIPTION OF THE PREFERRED EMBODIMENT(S)

Embodiments of the 3D video decoding apparatus according to the presentinvention are described in detail below with reference to the drawings.

First Embodiment

When there is an error in one of the left-eye and right-eye videoimages, a 3D video decoding apparatus according to the first embodimentof the present invention (i) provides 2D presentation in which onlynormal video images are displayed, in the case where an amount of dataunable to be decoded by the error (for example, the number of successivepictures with the errors) is large, and (ii) skips both the video imageswith the 3D presentation maintained, in the case where the amount ofdata unable to be decoded by the error is small. This allows the 3Dvideo decoding apparatus according to the first embodiment of thepresent invention to generate favorable video images when an erroroccurs.

First, a structure of a 3D video display system which includes the 3Dvideo decoding apparatus according to the first embodiment of thepresent invention is described.

FIG. 1 is a block diagram showing a structure of the 3D video displaysystem according to the first embodiment of the present invention.

A 3D video display system 10 shown in FIG. 1 includes a digitaltelevision 20, a digital video recorder 30, and shutter glasses 43. Thedigital television 20 and the digital video recorder 30 areinterconnected via a High-Definition Multimedia Interface (HDMI) cable40.

The digital video recorder 30 processes 3D video signals recorded on anoptical disc 41 such as a blu-ray disc (BD), and outputs the processed3D video signals to the digital television 20 via the HDMI cable 40.

The digital television 20 displays 3D video images which are representedby 3D video signals output from digital video recorder 30 and by 3Dvideo signals included in broadcast waves 42. For example, the broadcastwaves 42 include digital terrestrial television broadcasting or digitalsatellite broadcasting.

The digital video recorder 30 may process 3D video signals recorded on arecording medium (e.g., a hard disk drive or a non-volatile memory)other than the optical disc 41. Furthermore, the digital video recorder30 may process 3D video signals included in the broadcast waves 42 or 3Dvideo signals obtained through communications network such as theInternet. In addition, the digital video recorder 30 may also process 3Dvideo signals input from an external device to an external inputterminal (not shown) or the like.

Likewise, the digital television 20 may display video images representedby 3D video signals recorded on the optical disc 41 and other recordingmedia. Furthermore, the digital television 20 may display video imagesrepresented by 3D video signals obtained through communications networksuch as the Internet. In addition, the digital television 20 may displayvideo images which are represented by 3D video signals input from anexternal device other than the digital video recorder 30 to an externalinput terminal (not shown) or the like.

Furthermore, the digital television 20 may perform predeterminedprocessing on the obtained 3D video signals and display video imagesrepresented by the processed 3D video signals.

The digital television 20 and the digital video recorder 30 may also beinterconnected via a standardized cable other than the HDMI cable 40 orvia wireless communications network.

The digital video recorder 30 includes an input unit 31, a 3D videodecoding apparatus 100, and an HDMI communication unit 33.

The input unit 31 receives input video signals 111 recorded on theoptical disc 41.

The 3D video decoding apparatus 100 generates output video signals 117by decoding the input video signals 111.

The HDMI communication unit 33 outputs the output video signals 117generated by the 3D video decoding apparatus 100, to the digitaltelevision 20 via the HDMI cable 40.

The digital video recorder 30 may store the generated output videosignals 117 into a storage unit (such as a hard disk drive or anon-volatile memory) included in the digital video recorder 30, or mayalso store the generated output video signals 117 onto a recordingmedium (such as an optical disc) which can be inserted into and removedfrom the digital video recorder 30.

The digital television 20 includes an input unit 21, an HDMIcommunication unit 23, a 3D video decoding apparatus 100B, a displaypanel 26, and a transmitter 27.

The input unit 21 receives input video signals 56 included in thebroadcast waves 42.

The HDMI communication unit 23 receives output video signals 117provided by the HDMI communication unit 33, and outputs them as inputvideo signals 57.

The 3D video decoding apparatus 100B generates output video signals 58by decoding the input video signals 56 or the input video signals 57.

The display panel 26 displays video images which are represented by theoutput video signals 58 generated by the 3D video decoding apparatus100B.

The transmitter 27 controls the shutter glasses 43 using wirelesscommunications.

FIG. 2 shows an example of 3D video data. As shown in FIG. 2, the 3Dvideo data includes a left-eye image 170 l and a right-eye image 170 rwhich are alternately disposed.

FIG. 3 shows an example of the left-eye image 170 l and the right-eyeimage 170 r.

As shown in FIG. 3, objects included in the left-eye image 170 l and theright-eye image 170 r have a parallax which depends on a distance froman image capturing position to the objects.

The shutter glasses 43 are, for example, liquid crystal shutter glassesworn by a viewer, and include a left-eye liquid crystal shutter and aright-eye liquid crystal shutter. The transmitter 27 controls openingand closing of the left-eye liquid crystal shutter and the right-eyeliquid crystal shutter with the same timing of displaying the left-eyeimage 170 l and the right-eye image 170 r. Specifically, the transmitter27 opens the left-eye liquid crystal shutter of the shutter glasses 43and closes the right-eye liquid crystal shutter thereof while theleft-eye image 170 l is displayed. Furthermore, the transmitter 27closes the left-eye liquid crystal shutter of the shutter glasses 43 andopens the right-eye liquid crystal shutter thereof while the right-eyeimage 170 r is displayed. Thus, the left-eye image 170 l and theright-eye image 170 r selectively and respectively enter the left eyeand the right eye of the viewer.

It is to be noted that the method of selectively presenting the left-eyeimage 170 l and the right-eye image 170 r respectively to the left eyeand the right eye of the viewer is not limited to the method describedabove, and a method other than the above may be used.

For example, as shown in FIG. 4, left-eye lines 175 l and right-eyelines 175 r are arranged in a stripe pattern within each picture of the3D video data

In this case, the display panel 26 includes a left-eye polarizing filmformed on a left-eye pixel and a right-eye polarizing film formed on aright-eye pixel so that the left-eye lines 175 l and the right-eye lines175 r are subject to different polarizations (linear polarization,circular polarization, or the like). The shutter glasses 43 can bereplaced by polarized glasses having a left-eye polarizing filter and aright-eye polarizing filter which correspond to the above respectivepolarizations, so that the left-eye lines 175 l and the right-eye lines175 r enter the left eye and the right eye, respectively, of the viewer.

The arrangement pattern of the left-eye video images and the right-eyevideo images in the 3D video data may be other than the horizontallystriped pattern. For example, the left-eye video images and theright-eye video images may be arranged in a vertical stripe patternwithin each picture. Alternatively, the left-eye video images and theright-eye video images may be arranged in a checkered pattern within onepicture. Alternatively, the left-eye image 170 l and the right-eye image170 r may be arranged vertically or horizontally side by side within onepicture.

The 3D video decoding apparatus 100 according to the first embodiment ofthe present invention is described in detail below.

FIG. 5 is a block diagram showing a structure of the 3D video decodingapparatus 100 according to the first embodiment of the presentinvention. This 3D video decoding apparatus 100 includes a storing unit101, a decoding unit 103, an error determining unit 104, an outputdetermining unit 105, and an output unit 106.

The storing unit 101 stores the input video signals 111 and outputs themas input video signals 112.

A structure of the input video signal 112 (111) is described below.

For example, the input video signal 112 is stream data in compliancewith the H.264 MVC (multiview video coding)—Blu-ray disc (BD)three-dimensional (3D) standard.

FIG. 6 shows a structure of the input video signal 112.

The input video signal 112 is a transport stream (MPEG-2 TS) andincludes a plurality of TS packets, for example. Each of the TS packetsis a left-eye packet 151L obtained by coding a left-eye video signal, ora right-eye packet 151R obtained by coding a right-eye video signal.These left-eye packet 151L and right-eye packet 151R are arrangedalternately in the input video signal 112. Furthermore, the left-eyepacket 151L and the right-eye packet 151R which correspond to images tobe displayed at the same time point form a pair which is referred to asan access unit 152.

The images to be displayed at the same time point are, for example,images which are given the same presentation time stamp (PTS). It is tobe noted that, in the case where the left-eye image 170 l and theright-eye image 170 r are displayed alternately on the display panel 26as described above, the images with the same PTS are displayed not atthe same time but successively.

In addition, each of the TS packets is given an identifier whichindicates whether the packet is the left-eye packet 151L or theright-eye packet 151R. Thus, the 3D video decoding apparatus 100identifies, by referring to the identifier, a left-eye code signal 112L(the left-eye packet 151L) that is obtained by coding a video signal ofthe first view and is included in the input video signal 112, and aright-eye code signal 112R (the right-eye packet 151R) that is obtainedby coding a video signal of the second view different from the firstview and is included in the input video signal 112.

FIG. 7 shows a structure of the left-eye code signal 112L. The right-eyecode signal 112R has the same or like structure.

The left-eye code signals 112L include a plurality of sequence data 160.The sequence indicates a unit equivalent to a group of pictures (GOP) inthe MPEG-2 standard.

The sequence data 160 includes a sequence header 161 and a plurality ofpicture data 162. The sequence header 161 includes control informationwhich is common to the plurality of picture data 162 included in thecorresponding sequence data 160.

Each of picture data 162 includes a picture header 163 and pixel data164. The picture header 163 includes control information on the pixeldata 164 included in the corresponding picture data 162. The pixel data164 is data obtained by coding data of a single picture (which ishereinafter referred to also as a coded picture).

It is to be noted that each of the TS packets shown in FIG. 6 isfixed-length data and corresponds to part of one piece of the picturedata 162 or corresponds to one or more pieces of the picture data 162.

With reference to FIG. 5 again, further description is set forth below.

The decoding unit 103 generates a left-eye decode signal 113L bydecoding the left-eye code signal 112L. Furthermore, the decoding unit103 generates a right-eye decode signal 113R by decoding the right-eyecode signal 112R. The decoding unit 103 then outputs decoded videosignals 113 including the left-eye decode signal 113L and the right-eyedecode signal 113R.

The left-eye code signal 112L is a base view that is decoded using theleft-eye code signal 112L only. The right-eye code signal 112R is anindependent view that is decoded using the right-eye code signal 112Rand the left-eye code signal 112L.

Specifically, the decoding unit 103 generates quantized coefficients byvariable-length decoding of the left-eye code signal 112L and theright-code signal 112R. Next, the decoding unit 103 generates orthogonaltransform coefficients (DCT coefficients) by inverse quantizing thegenerated quantized coefficients. Next, the decoding unit 103 generatesprediction errors by inverse orthogonal transforming the generatedorthogonal transform coefficients. In the meantime, the decoding unit103 generates a predictive picture by motion compensation using adecoded reference picture. Next, the decoding unit 103 generates adecoded picture (decoded video signals 113) by adding up the generatedprediction errors and predictive picture. Furthermore, the decoding unit103 stores, into a memory unit, the generated decoded picture as areference picture which is to be used in a decoding process of asucceeding picture.

FIG. 8 shows a reference relation in decoding of coded pictures.

The left-eye code signals 112L and the right-eye code signals 112Rinclude coded I-pictures, coded P-pictures, and coded B-pictures. Thecoded I-picture, the coded P-picture, and the coded B-picture are codedpictures obtained by coding an I-picture, a P-picture, and a B-picture,respectively. The I-picture is a picture coded using data within thepicture itself only. The P-picture and the B-pictures are pictures codedusing another I-picture or P-picture.

In other words, the coded I-picture is decoded using data within thecoded picture itself only. The coded P-picture and the coded B-pictureare decoded using data within the coded pictures themselves and datawithin other decoded pictures. In the following description, the use ofthe decoded picture as a reference picture may be referred to asreferring to the decoded picture.

In FIG. 8, the coded picture 12 is the coded I-picture, the codedpictures P2 and P5 are the coded P-pictures, and the coded pictures B0,B1, B3, and B4 are the coded B-pictures. In addition, an arrow shown inFIG. 8 indicates a decoded picture to which each of the coded picturesrefers.

As shown in FIG. 8, the coded P-pictures and coded-B pictures includedin the left-eye code signals 112L refer to only the decoded I-picturesand decoded P-pictures included in the left-eye decode signals 113L. Thecoded P-pictures and coded B-pictures included in the right-eye codesignals 112R refer to the decoded I-pictures and decoded P-picturesincluded in the right-eye decode signals 113R, and the decoded picturesincluded in the same access unit 152 as the coded picture among thedecoded pictures included in the left-eye decode signals 113L. Forexample, in the example shown in FIG. 8, the coded picture P2 includedin the right-eye code signals 112R refers to the decoded picture 12included in the same access unit as the coded picture P2.

While the left-eye decode signal 113L is basically referred to by theright-eye code signal 112R within the same access unit 152 as above,such a reference within the access unit 152 is not carried out whenthere is a large difference in video images between the left-eye decodesignals 113L and the right-eye decode signals 113R. In addition, thereference is basically carried out only within the access unit 152. Thismeans that the coded picture in the right-eye code signals 112R does notrefer to the decoded picture in the left-eye decode signals 113L whichis included in a different access unit 152.

Thus, the decoding unit 103 decodes the left-eye code signals 112L withreference to the decoded left-eye decode signals 113L only. Furthermore,the decoding unit 103 decodes the right-eye code signals 112R withreference to the decoded left-eye decode signals 113L and right-eyedecode signals 113R.

The error determining unit 104 determines whether or not the decodingunit 103 can correctly decode the left-eye code signals 112L and whetheror not the decoding unit 103 can correctly decode the right-eye codesignals 112R. Specifically, the error determining unit 104 determineswhether or not the decoding unit 103 can correctly decode the respectivecoded pictures included in the left-eye code signals 112L and theright-eye code signals 112R. That is, the error determining unit 104determines whether or not the input video signals 112 (111) include dataloss or corruption (which is hereinafter referred to as an error) due toscratches or dirt on a BD disc etc., a packet loss by a distributionerror in the network, and the like causes.

For example, the error determining unit 104 determines that there is anerror in the video signals, when one of the left-eye code signal 112Land the right-eye code signal 112R lacks data of a picture whichcorresponds to a picture included in the other of the left-eye codesignal 112L and the right-eye code signal 112R, and when a data valueand a format are out of predetermined normal ranges. It is to be notedthat the error determining unit 104 may perform the error determinationon a per picture basis and may also perform it based on other units(e.g., for each slice, macroblock, or set of multiple pictures).

The output determining unit 105 determines whether to skip (i.e., notoutput) both the left-eye decode signal 113L and the right-eye decodesignal 113R or to output only the decode signal in which there is noerror, when the error determining unit 104 determines that there is anerror in only one of the left-eye code signal 112L and the right-eyecode signal 112R which are included in the same access unit 152.

Here, as described above, the right-eye code signal 112R is decoded byreferring to the left-eye code signal 112L. Thus, when there is an errorin the left-eye code signal 112L, the right-eye code signal 112R mayalso not be correctly decoded.

The output determining unit 105 therefore determines that both theleft-eye decode signal 113L and the right-eye decode signal 113R are tobe skipped, when there is an error in the left-eye code signal 112L.When there is an error in the right-eye code signal 112R, the outputdetermining unit 105 determines that only the left-eye decode signal113L in which there is no error is to be output.

In addition, the output determining unit 105 further determines whetheror not the data amount of the right-eye code signal 112R which isdetermined by the error determining unit 104 as being unable to becorrectly decoded (for example, the number of coded pictures witherrors) is equal to or greater than the first predetermined threshold.When the number of successive coded pictures with errors (which picturesare hereinafter referred to as error pictures) is smaller than the firstthreshold, the output determining unit 105 determines that neither theleft-eye decode signal 113L nor the right-eye decode signal 113R is tobe output, while, when the number of successive error pictures is equalto or greater than the first threshold, the output determining unit 105determines that only the left-eye decode signal 113L is to be output.

The output unit 106 outputs the left-eye decode signals 113L and theright-eye decode signals 113R as the output video signals 117. However,the output unit 106 does not output any or one of the left-eye decodesignal 113L and the right-eye decode signal 113R determined by theoutput determining unit 105 as being skipped (i.e., not output). Here,skipping indicates not outputting data of a corresponding decodedpicture or outputting the same data as the immediately preceding decodedpicture in the video signals of the same view.

The decoding unit 103 does not perform the decoding process on the codedpicture which corresponds to the decoded picture determined by theoutput determining unit 105 as being skipped. It may also be possiblethat the decoding unit 103 performs all or part of the decoding processand the output unit 106 does not perform only the output process.

The following describes a flow of operation of the 3D video decodingapparatus 100.

FIG. 9 is a flowchart showing a decoding process of the 3D videodecoding apparatus 100.

As shown in FIG. 9, first, the error determining unit 104 determineswhether or not there is an error in the input video signals 112 (S101).

Where there is no error (No in S101), the output unit 106 outputs theleft-eye decode signals 113L and the right-eye decode signals 113R asthe output video signals 117 (S102). This allows the display panel 26 todisplay the 3D video images.

On the other hand, there is an error in the left-eye code signals 112L(Yes in S101 and Yes in S103), the output unit 106 outputs the outputvideo signals 117 with the left-eye decode signals 113L and theright-eye decode signals 113R skipped (S104). This causes the displaypanel 26 to display the immediately preceding 3D video images again.

When there is an error in the right-eye code signals 112R (No in S103),then the output determining unit 105 determines whether or not the datamount with the errors is equal to or greater than the first threshold(S105).

The following describes a specific example of the method in which theoutput determining unit 105 determines an amount of data with errors.

FIGS. 10 and 11 show examples of the input video signals 112 and theoutput video signals 117 in the case where there is an error in theright-eye code signals 112R. The output video signals 117 include theleft-eye output signals 117L and the right-eye output signals 117R. Theleft-eye output signals 117L correspond to the left-eye decode signals113L, and the right-eye output signals 117R correspond to the right-eyedecode signals 113R.

For example, the output determining unit 105 calculates a differencebetween the PTS assigned to the initial coded picture 170, not yetdecoded, of the left-eye code signals 112L stored in the storing unit101, and the PTS assigned to the initial coded picture 171, not yetdecoded, of the right-eye code signals 112R stored in the storing unit101. When the calculated difference is equal to or greater than thesecond threshold, the output determining unit 105 determines that theamount of data with errors is equal to or greater than the firstthreshold, while, when the calculated difference is smaller than thesecond threshold, the output determining unit 105 determines that theamount of data with errors is smaller than the first threshold.

With an underflow of the right-eye code signals 112R, the outputdetermining unit 105 may determine that the amount of data with errorsis equal to or greater than the first threshold.

It may also be possible that the output determining unit 105 performsthe same or like determination using the left-eye decode signals 113Land the right-eye decode signals 113R stored in a memory unit (notshown) in which the decoded video signals 113 are stored.

As shown in FIG. 10, when the amount of data with errors is smaller thanthe first threshold (No in S105), the output video signals 117 areoutput with the left-eye decode signals 113L and the right-eye decodesignals 113R skipped (S104).

On the other hand, as shown in FIG. 11, when the amount of data witherrors is equal to or greater than the first threshold (Yes in S105),the output video signals 117 are output with the left-eye decode signals113L included and the right-eye decode signals 113R skipped (S106). Thiscauses the display panel 26 to display only the left-eye image 170 l in2D.

Although not shown in FIG. 9, when there are errors in both the left-eyecode signals 112L and the right-eye code signals 112R, the outputdetermining unit 105 outputs the output video signals 117 with theleft-eye decode signals 113L and the right-eye decode signals 113Rskipped.

As above, the 3D video decoding apparatus 100 according to the firstembodiment of the present invention maintains the 3D presentation byskipping both the left-eye image and the right-eye image when the amountof data with errors is smaller than the first threshold. This allows the3D video decoding apparatus 100 to prevent an instantaneous change from3D to 2D presentation when the amount of data with errors is small.Thus, the 3D video decoding apparatus 100 is capable of generatingfavorable video images when there is an error.

Furthermore, the 3D video decoding apparatus 100 according to the firstembodiment of the present invention provides the 2D presentation whenthe amount of data with errors is equal to or greater than the firstthreshold. This allows the 3D video decoding apparatus 100 to avoid along video freeze when the amount of data with errors is large. Thus,the 3D video decoding apparatus 100 is capable of generating favorablevideo images when there is an error.

Furthermore, the 3D video decoding apparatus 100 according to the firstembodiment of the present invention skips both the left-eye decodesignal 113L and the right-eye decode signal 113R when there is an errorin the left-eye code signal 112L. This makes it possible to prevent theerror occurred in the left-eye code signal 112L to pass on to theright-eye decode signal 113R that is generated with reference to theleft-eye decode signal 113L. Furthermore, the 3D video decodingapparatus 100 provides the 2D presentation when there is an error in theright-eye code signal 112R. This allows the 3D video decoding apparatus100 to avoid frequent freezes of video. Thus, the 3D video decodingapparatus 100 is capable of generating favorable video images when thereis an error.

While the above description gives an example where the left-eye codesignals 112L serve as a base view and the right-eye code signals 112Rserve as a dependent view, it may also be possible that the right-eyecode signals 112R serve as a base view and the left-eye code signals112L serve as a dependent view.

Furthermore, while the above description gives an example where the 3Dvideo decoding apparatus 100 processes the respective left-eye andright-eye video images of the two views, it may process video images ofthree or more views. That is, there may be a plurality of dependentviews.

Furthermore, while the above description gives an example where theright-eye code signal 112R is decoded with reference to the left-eyedecode signal 113L, the right-eye code signal 112R and the left-eye codesignal 112L may be signals which are decoded with reference to their owndecode signals only. Even in this case, the same effects as above can beobtained by switching, according to the amount of data with errors,between skipping of both the decode signals and outputting of only thedecode signal with no error.

Furthermore, the order of the processes shown in FIG. 9 is an example,and the respective steps may be performed in other orders. For example,the determining process of Step S103 and the determining process of StepS105 may be replaced in the order or may partially be performed at thesame time.

Second Embodiment

The first embodiment has described the processing of the 3D videodecoding apparatus 100 mainly in the case where there is a loss of datawhich corresponds to a plurality of pictures. The second embodiment ofthe present invention describes the operation of the 3D video decodingapparatus 100 mainly in the case where there is an error in data of asingle coded picture due to dirt, scratches, and the like on BD discs.

FIG. 12 is a flowchart showing a decoding process of the 3D videodecoding apparatus 100 according to the second embodiment of the presentinvention.

As shown in FIG. 12, first, the error determining unit 104 determineswhether or not there is an error in the input video signals 112 (S201).Here, the error determining unit 104 determines the error on a per codedpicture basis and at the same time, determines, for each of the slicesincluded in the error picture, whether or not the slice can be correctlydecoded. It is to be noted that the error determining unit 104 maydetermine, for each of subregions (e.g., one or more macroblocks),except for the slices, of the coded picture, whether or not thesubregion can be correctly decoded.

Where there is no error (No in S201), the output unit 106 outputs theleft-eye decode signals 113L and the right-eye decode signals 113R asthe output video signals 117 (S202). This allows the display panel 26 todisplay the 3D video images.

On the other hand, there is an error (Yes in S201), then the outputdetermining unit 105 determines whether the coded picture with the erroris a reference coded picture or a non-reference coded picture (S203).The reference coded picture indicates a coded picture which is decodedto generate a decoded picture which is referred to when the decodingunit 103 decodes another coded picture included in the video signals ofthe same view, and is specifically the coded I picture and the coded Ppicture. The non-reference coded picture indicates a coded picture whichis decoded to generate a decoded picture which is not referred to whenthe decoding unit 103 decodes another coded picture included in thevideo signals of the same view, and is specifically the coded B picture.

FIG. 13 shows an example of the input video signals 112 and the outputvideo signals 117 in the case where there is an error in thenon-reference coded picture (an error picture 180).

As shown in FIG. 13, when there is an error in the non-reference codedpicture, the output unit 106 outputs the output video signals 117 withthe left-eye decode signals 113L and the right-eye decode signals 113Rskipped until the next decoded picture (S204). In other words, theoutput unit 106 skips an error decoded picture corresponding to theerror picture, and a decoded picture which is included in the sameaccess unit 152 as the error decoded picture and included in the videosignals of the other views.

On the other hand, when there is an error in the reference coded picture(Yes in S203), then the output determining unit 105 determines which, ofthe left-eye code signal 112L and the right-eye code signal 112R,includes the error (S205).

When there is an error in the reference coded picture, referring to theerror decoded picture by a succeeding picture may cause propagation ofthe error. The output determining unit 105 therefore skips the decodedpictures until the next sequence, when the reference coded pictureincludes an error.

FIG. 14 shows an example of the input video signals 112 and the outputvideo signals 117 in the case where there is an error in the referencecoded picture (an error picture 181) of the right-eye code signals 112R.

As shown in FIG. 14, when there is an error in the reference codedpicture of the right-eye code signals 112R (No in S205), the output unit106 outputs the output video signals 117 with the right-eye decodesignals 113R skipped until the next sequence (S206).

On the other hand, when there is an error in the reference coded pictureof the left-eye code signals 112L (Yes in S205), then the outputdetermining unit 105 determines whether or not the slice with the error(which slice is hereinafter referred to as an error slice) is includedin a reference area 187 of the coded picture, included in the sameaccess unit 152 as the error picture, of the right-eye code signals 112R(S207).

FIG. 15 shows a relation between an error slice 186 and the referencearea 187. The reference area 187 is an area which is associated witheach of the coded pictures included in the right-eye code signals 112Rand is located inside the decoded picture of the left-eye decode signals113L which is referred to by the corresponding coded picture. That is,the reference area 187 is an area inside the error decoded picture whichis referred to by the coded picture of the right-eye code signals 112Rwhich is included in the same access unit 152 as the error picture.Reference area designating information indicating this reference area187 is included in the input video signals 112. Specifically, thisreference area designating information is parallel decoding informationSEI in the H.264 MVC standard.

The decoding unit 103 decodes the coded picture included in theright-eye code signals 112R, with reference to the reference area 187included in the decoded picture of the left-eye code signals 113L whichis included in the same access unit 152 as the coded picture.

As shown in FIG. 15, even in the case where there is an error in theleft-eye code signals 112L, when the error slice 186 is not included inthe reference area 187, in other words, when the reference area 187includes only normal slices 185 that can be correctly decoded, the codedpicture of the right-eye code signals 112R which refers to the errordecoded picture can be correctly decoded.

FIG. 16 shows an example of the input video signals 112 and the outputvideo signals 117 in the case where the error slice 186 is not includedin the reference area 187.

As shown in FIG. 16, when the error slice 186 of the error picture 182is not included in the reference area 187 (No in S207), the output unit106 outputs the output video signals 117 with the left-eye decodesignals 113L skipped until the next sequence (S208).

FIG. 17 shows an example of the input video signals 112 and the outputvideo signals 117 in the case where the error slice 186 is included inthe reference area 187.

As shown in FIG. 17, when the error slice 186 of the error picture 182is included in the reference area 187 (Yes in S207), the output unit 106outputs the output video signals 117 with both the left-eye decodesignals 113L and the right-eye decode signals 113R skipped until thenext sequence (S209).

As above, the 3D video decoding apparatus 100 according to the secondembodiment of the present invention skips both the left-eye decodesignal 113L and the right-eye decode signal 113R when there is an errorin the left-eye code signal 112L. This makes it possible to prevent theerror occurred in the left-eye code signal 112L to pass on to theright-eye decode signal 113R that is generated with reference to theleft-eye decode signal 113L. Furthermore, the 3D video decodingapparatus 100 provides the 2D presentation when there is an error in theright-eye code signal 112R. This allows the 3D video decoding apparatus100 to avoid frequent freezes of video. Thus, the 3D video decodingapparatus 100 is capable of generating favorable video images when thereis an error.

Furthermore, when the error picture is the non-reference coded picture,the 3D video decoding apparatus 100 according to the second embodimentof the present invention skips both the left-eye decode signals 113L andthe right-eye decode signals 113R until the next decoded picture. Thisallows the 3D video decoding apparatus 100 to skip the minimum number ofpictures. When the error picture is the reference coded picture, the 3Dvideo decoding apparatus 100 skips both the left-eye decode signals 113Land the right-eye decode signals 113R or the decode signals with theerrors until the next sequence. This allows the 3D video decodingapparatus 100 to prevent the error from passing on to a succeedingdecoded picture.

Furthermore, when skipping until the next decoded picture, the 3D videodecoding apparatus 100 according to the second embodiment of the presentinvention skips both the respective-eye decode signals. This allows the3D video decoding apparatus 100 to prevent video from being displayed in2D only for a moment. When skipping decoding until the next sequence,the 3D video decoding apparatus 100 provides the 2D presentation. Thisallows the 3D video decoding apparatus 100 to avoid a long freeze ofvideo.

Furthermore, even in the case where there is an error in the left-eyecode signals 112L, the 3D video decoding apparatus 100 according to thesecond embodiment of the present invention outputs only the right-eyedecode signals 113R when the error slice 186 is not included in thereference area 187 to which the corresponding right-eye code signal 112Rrefers. This allows the 3D video decoding apparatus 100 to avoidfrequent freezes of video.

Thus, the 3D video decoding apparatus 100 according to the secondembodiment of the present invention is capable of generating favorablevideo images when there is an error.

While the above describes the 3D video decoding apparatus 100 whichselects the decode signals to be output or skipped, according to theresult of a plurality of determining processes as described above, the3D video decoding apparatus 100 may select the decode signals to beoutput or skipped, according to the result of at least one of the abovedetermining processes.

Specifically, it is sufficient that the 3D video decoding apparatus 100according to the second embodiment of the present invention performs atleast one of the following processes: (i) a process of skipping both theleft-eye decode signals 113L and the right-eye decode signals 113R whenthere is an error in the left-eye code signals 112L, and providing the2D presentation when there is an error in the right-eye code signals112R; (ii) a process of skipping both the left-eye decode signals 113Land the right-eye decode signals 113R until the next decoded picture,when the error picture is the non-reference coded picture, and skippingboth the left-eye decode signals 113L and the right-eye decode signals113R or the decode signals with the errors until the next sequence, whenthe error picture is the reference coded picture; (iii) a process ofskipping both the respective-eye decode signals when skipping thesignals until the next decoded picture in the above (ii), and providingthe 2D presentation when skipping the signals until the next sequence inthe above (ii); and (iv) a process of outputting only the right-eyedecode signals 113R when the error slice 186 is not included in thereference area 187 to which the corresponding right-eye code signals112R refer, even when there is an error in the left-eye code signals112L.

Furthermore, the order of the processes shown in FIG. 12 is an example,and other orders of the processes are applicable as long as they canlead to the same or like effects. Part of the processes may be performedat the same time.

Third Embodiment

A 3D video decoding apparatus 200 according to the third embodiment ofthe present invention complements the header information which includesan error, using the other header information included in the videosignals of the same view or with the header information included in thevideo signals of a different view.

The 3D video decoding apparatus 200 according to the third embodiment ofthe present invention can be applied to the 3D video display system 10shown in FIG. 1, as in the case of the 3D video decoding apparatus 100according to the first embodiment.

FIG. 18 is a block diagram showing a structure of the 3D video decodingapparatus 200 according to the third embodiment of the presentinvention. In the figure, components common with FIG. 5 have the samenumerals.

This 3D video decoding apparatus 200 includes the storing unit 101, thedecoding unit 103, an error determining unit 204, a header informationstoring unit 207, and the output unit 106.

The storing unit 101 stores the input video signals 111 and outputs themas the input video signals 112.

The header information storing unit 207 stores header information. Theheader information is control information which is included in each ofthe sequence header 161 and the picture header 163 shown in FIG. 7.

The error determining unit 204 determines whether or not the sequenceheader 161 and the picture header 163 are normal (whether they includeerrors).

For example, when the sequence header 161 or the picture header 163 isnot present and when a data value and a format of the sequence header161 or the picture header 163 are out of predetermined normal ranges,the error determining unit 204 determines that there is an error in thecorresponding one of the sequence header 161 and the picture header 163.Specifically, when a next start code is detected during analysis of theheader information or when the payload length of the TS packet is out ofa predetermined range or when IP packet loss occurs during distributionin the network, the error determining unit 204 determines that there isan error in such header information.

Furthermore, the error determining unit 204 determines, using the headerinformation stored in the header information storing unit 207, whetheror not the current header information is appropriate. Specifically, whenthe common header information in the video signals of the same view isnot the same as the header information of the preceding sequence or whena difference between the present header information and the precedingheader information is equal to or greater than the third threshold, theerror determining unit 204 determines that there is an error in thecurrent header information.

When the error determining unit 204 determines that there is an error inthe header information, the complementing unit 208 complements theheader information which includes the error, using the preceding headerinformation included in the video signals of the same view or the headerinformation included in the video signals of a different view at thecorresponding presentation time point.

The decoding unit 103 generates the left-eye decode signals 113L bydecoding the left-eye code signals 112L using the header informationincluded therein. Furthermore, the decoding unit 103 generates theright-eye decode signals 113R by decoding the right-eye code signals112R using the header information included therein. In addition, thedecoding unit 103 generates the decode video signals 113 including theleft-eye decode signals 113L and the right-eye decode signals 113R. Whenthere is an error in the header information of the input video signals112, the decoding unit 103 decodes the input video signals 112 using theheader information complemented by the complementing unit 208.

The output unit 106 outputs the left-eye decode signals 113L and theright-eye decode signals 113R as the output video signals 117.

The following describes an operation of the 3D video decoding apparatus200.

FIG. 19 shows a complementing process of the 3D video decoding apparatus200. FIG. 20 is a flowchart showing the complementing process of the 3Dvideo decoding apparatus 200.

As shown in FIG. 19, the input video signal 112 includes the left-eyecode signal 112L, the first right-eye code signal 112R1, and the secondright-eye code signal 112R2. That is, the right-eye code signal 112Rincludes the first right-eye code signal 112R1 and the second right-eyecode signal 112R2. These first right-eye code signal 112R1 and secondright-eye code signal 112R2 are both dependent views. For example, thefirst right-eye code signal 112R1 and the second right-eye code signal112R2 are right-eye vide signals which have different parallaxes (shiftamounts) with respect to the left-eye code signal 112L.

The decoding unit 103 generates the first right-eye decode signals bydecoding the first right-eye code signals 112R1 using the headerinformation included therein. The decoding unit 103 generates the secondright-eye decode signals by decoding the second right-eye code signals212R2 using the header information included therein. The decoding unit103 then outputs the decoded video signals 113 including the left-eyedecode signals 113L, the first right-eye decode signals and the secondright-eye decode signals. The output unit 106 outputs the left-eyedecode signals 113L, the first right-eye decode signals, and the secondright-eye decode signals, as the output video signals 117.

It is also possible that the decoding unit 103 generates the right-eyedecode signal 113R by decoding one of the first right-eye code signal112R1 and the second right-eye code signal 112R2 selectively accordingto a control signal or the like from outside. In this case, the outputunit 106 outputs, as the output video signals 117, the left-eye decodesignals 113L and the right-eye decode signals 113R generated by thedecoding unit 103.

Furthermore, each of the left-eye code signal 112L, the first right-eyecode signal 112R1, and the second right-eye code signal 112R includes asequence parameter set (SPS) 190. In addition, each of the firstright-eye code signal 112R1 and the second right-eye code signal 112R2includes a subset SPS (SSPS) 191. These SPS 190 and SSPS 191 areincluded in the sequence header 161 shown in FIG. 7.

The SPS 190 is control information which is common to the plurality ofpicture data 162 included in the corresponding sequence data 160. TheSSPS 191 is information indicating a relation of the video signals amongthe views (a relation among the left-eye code signal 112L, the firstright-eye code signal 112R1, and the second right-eye code signal112R2).

As shown in FIG. 20, when there is an error in the SPS 190 (Yes inS301), the complementing unit 208 reads, from the header informationstoring unit 207, the SPS 190 of another normal sequence included in thecode signals of the same view, and replaces the SPS 190 which includesthe error, by the read normal SPS 190 (S302).

It is to be noted that the complementing unit 208 may use not only theSPS 190 but also other information in other normal sequences or picturesas long as the information is the same within the code signals of thesame view. The information which is the same within the code signals ofthe same view is, for example, priority_id and view_id in the H.264 MVCstandard. Here, priority_id indicates a priority in decoding the codesignals of the corresponding view. In other words, priority_id indicatesthe order of decoding the code signals of a plurality of views.Furthermore, view_id is information for identifying the code signals ofthe corresponding view. For example, “0” is assigned to a base viewwhile “1, 2 . . . ” are assigned to dependent views.

Furthermore, when there is an error in the SSPS 191 (Yes in S303), thecomplementing unit 208 replaces the SSPS 191 which includes the error,by the normal SSPS 191 included in the code signals of a different view(S304).

Furthermore, when there is an error in the information which is the samewithin the access unit 152 (Yes in S305), the complementing unit 208replaces the information which includes the error, by the normalinformation which is included in the video signals of a different viewand included in the same access unit 152 (S306).

The information which is the same within the access unit is, forexample, non_idr_flag and anchor_pic_flag in the H.264 MVC standard, andnal_unit_type and temporal_id in the BD standard.

Here, non_idr_flag is information indicating whether or not the pictureis an IDR picture. The IDR picture is a kind of I picture, and it isforbidden that a picture succeeding the IDR picture refers to a picturepreceding the IDR picture. This non_idr_flag indicates that a picture ofat least a base view is an IDR picture, and it is therefore not requiredthat a picture of a dependent view is an IDR picture.

Here, anchor_pic_flag is information indicating whether or not thepicture is an initial picture of a sequence.

Furthermore, nal_unit_type is information indicating a data attribute.For example, nal_unit_type indicates whether the data is headerinformation or an IDR picture.

Furthermore, temporal_id is an identifier indicating the order ofdecoding a plurality of pictures. To each of successive pictures, serialnumbered temporal_id is assigned.

As above, when there is an error in the header information which iscommon to different views and within the same access unit 152, the 3Dvideo decoding apparatus 200 according to the third embodiment of thepresent invention complements the header information which includes theerror, using the header information of the coded picture which is of adifferent view and within the same access unit 152. This allows the 3Dvideo decoding apparatus 200 to appropriately complement the headerinformation which includes the error, and thereby generate favorablevideo images when there is an error.

Furthermore, when there is an error in the header information which iscommon within the video signals of the same view, the 3D video decodingapparatus 200 according to the third embodiment of the present inventioncomplements the header information which includes the error, using thenormal header information of a different sequence or coded picture whichis included in the video signals of the same view. This allows the 3Dvideo decoding apparatus 200 to appropriately complement the headerinformation which includes the error, and thereby generate favorablevideo images when there is an error.

Furthermore, when there is an error in the SSPS 191, the 3D videodecoding apparatus 200 according to the third embodiment of the presentinvention complements the SSPS 191 which includes the error, using theSSPS 191 of a different dependent view. This allows the 3D videodecoding apparatus 200 to appropriately complement the SSPS 191 whichincludes the error, and thereby generate favorable video images whenthere is an error.

It is to be noted that the complementing unit 208 may not only replacethe header information which includes an error, by the normal headerinformation which is included in the video signals of the same ordifferent view, but also complement the header information whichincludes an error, using the normal header information.

Specifically, when nal_unit_type of the base view indicates that thepicture is an IDR picture, the complementing unit 208 is capable ofcomplementing non_idr_flag of the picture of the base view. In addition,the complementing unit 208 is capable of complementing non_idr_flag ofthe dependent view by assigning the value of complemented non_idr_flagof the base view.

Furthermore, the complementing unit 208 is capable of complementing alsoanchor_pic_flag of the dependent view using stream information(indicating a position of the SPS 190 and indicating that the picture isan I picture) of the base view.

Furthermore, while the above description gives an example where the 3Dvideo decoding apparatus 200 processes the video signals of the threeviews, it may process video signals of four or more views.

Furthermore, the above describes the 3D video decoding apparatus 200according to the third embodiment of the present invention whichperforms the following processes: (i) when there is an error in theheader information which is common to different views and within thesame access unit 152, a process of complementing the header informationwhich includes the error, using the header information of the picturewhich is of a different view and within the same access unit 152; (ii)when there is an error in the header information which is common withinthe video signals of the same view, a process of complementing theheader information which includes the error, using the normal headerinformation of a different sequence or picture which is included in thevideo signals of the same view; and (iii) when there is an error in theSSPS 191, a process of complementing the SSPS 191 which includes theerror, using the SSPS 191 of a different dependent view, but the 3Dvideo decoding apparatus 200 may perform one or more of the aboveprocesses.

Furthermore, the order of the processes shown in FIG. 20 is an example,and other orders of the processes are applicable as long as they canlead to the same or like effects. Part of the processes may be performedat the same time.

Fourth Embodiment

A 3D video decoding apparatus 300 according to the fourth embodiment ofthe present invention selects one of the 2D presentation and the 3Dpresentation, according to the playback mode (which includes the normalplayback and the trick playback).

The 3D video decoding apparatus 300 according to the fourth embodimentof the present invention can be applied to the 3D video display system10 shown in FIG. 1, as in the case of the 3D video decoding apparatus100 according to the first embodiment.

FIG. 21 is a block diagram showing a structure of the 3D video decodingapparatus 300 according to the fourth embodiment of the presentinvention. In the figure, components common with FIG. 5 have the samenumerals.

This 3D video decoding apparatus 300 decodes the input video signals 111and outputs the output video signals 117 which are reproduced in thenormal playback mode or in the trick play mode. This 3D video decodingapparatus 300 includes the storing unit 101, the decoding unit 103, aplayback mode receiving unit 304, a depth determining unit 305, and theoutput unit 106.

The storing unit 101 stores the input video signals 111 and outputs themas the input video signals 112.

The playback mode receiving unit 304 receives, based on playback modedesignating signals 315, the playback mode designated by user operationor the like. The playback mode includes a normal playback mode and atrick play mode. The trick play mode includes a fast playback mode (ahigh-speed playback mode), a slow playback mode, a frame-by-frameplayback mode, a reverse playback mode, a reverse slow playback mode,and a reverse frame-by-frame mode. The fast playback mode includes aplurality of fast playback modes at different speeds (which include a1.3× playback mode, a 1.6× playback mode, and a 2× playback mode).

Furthermore, the fast playback mode is a mode in which the pictures inthe normal playback mode are partly skipped when displayed. The reverseplayback mode (the reverse slow playback mode and the reverseframe-by-frame playback mode) is a mode in which the pictures in thenormal playback mode are displayed in reverse order.

The decoding unit 103 decodes the left-eye code signals 112L and theright-eye code signals 112R and thereby generate the decode videosignals 113 including the left-eye decode signals 113L and the right-eyedecode signals 113R. Furthermore, the decoding unit 103 performs adecoding process according to the playback mode received by the playbackmode receiving unit 304 and thereby generates the decode video signals113 which are in the normal playback mode or in the trick play mode.

The output unit 106 outputs the left-eye decode signals 113L and theright-eye decode signals 113R as the output video signals 117.Furthermore, according to the playback mode received by the playbackmode receiving unit 304, the output unit 106 selects to output theleft-eye decode signals 113L and the right-eye decode signals 113R asthe output video signals 117 or to output only the left-eye decodesignals 113L as the output video signals 117.

The depth determining unit 305 calculates, using the informationincluded in the input video signals 112, an amount of change in thedepth of the decode video signals 113 (that is a depth-wise position inthe 3D presentation represented by the left-eye decode signals 113L andthe right-eye decode signals 113R). Furthermore, the depth determiningunit 305 determines whether or not the calculated amount of change isequal to or greater than the fourth predetermined threshold.

FIG. 22 is a flowchart showing a decoding process of the 3D videodecoding apparatus 300.

As shown in FIG. 22, when the playback mode received by the playbackmode receiving unit 304 is the normal playback mode (Yes in S401), thedecoding unit 103 decodes the left-eye code signals 112L and theright-eye code signals 112R and thereby generates the left-eye decodesignals 113L and the right-eye decode signals 113. The output unit 106outputs the left-eye decode signals 113L and the right-eye decodesignals 113R as the output video signals 117 (S402). This allows thedisplay panel 26 to display the 3D video images in the normal playbackmode.

On the other hand, when the playback mode received by the playback modereceiving unit 304 is the reverse playback mode, the reverse slowplayback mode, or the reverse frame-by-frame mode, the decoding unit 103decodes only the left-eye code signals 112L and thereby generates theleft-eye decode signals 113L. The output unit 106 outputs only theleft-eye decode signals 113L as the output video signals 117 (S407).This allows the display panel 26 to display the 2D video images in thereverse playback mode.

When the playback mode received by the playback mode receiving unit 304is the fast playback mode which is at 1.5× or higher speed (Yes in S404and Yes in S405), the decoding unit 103 decodes only the left-eye codesignals 112L and thereby generates the left-eye decode signals 113L. Theoutput unit 106 outputs only the left-eye decode signals 113L as theoutput video signals 117 (S407). This allows the display panel 26 todisplay the 2D video images in the fast playback mode.

When the playback mode received by the playback mode receiving unit 304is the fast playback mode which is at speed slower than 1.5× (Yes inS404 and No in S405), then the depth determining unit 305 calculates theamount of change in the depth of the input video signals 112 anddetermines whether the calculated amount of change is equal to orgreater than the fourth threshold or is smaller than the fourththreshold (S406).

Specifically, the input video signals 112 include depth informationindicating a depth of the decode video signals 113. When there are aplurality of depths in an image to be displayed; for example, in thecase of an image where the upper part is to be displayed in back whilethe lower part is to be displayed in front, the input video signals 112may include separate depth information for each of the regions obtainedby sectioning the image. With reference to this depth information, thedepth of a subtitle is corrected so that the video (the primary videoimage) does not penetrate the subtitle.

Using this depth information, the depth determining unit 305 determinesthe amount of change in the depth of the decode video signals 113.Specifically, when the amount of change in the depth indicated by thedepth information is equal to or greater than the fifth threshold, thedepth determining unit 305 determines that the amount of change in thedepth of the decode video signals 113 is equal to or greater than thefourth threshold, and when the amount of change in the depth indicatedby the depth information is smaller than the fifth threshold, the amountof change in the depth of the decode video signals 113 is smaller thanthe fourth threshold.

The amount of change indicates, for example, the maximum or averagevalue of the amounts of change among the successive pictures in part orall of the playback sections.

When the depth determining unit 305 determines that the amount of changein the depth is equal to or greater than the fourth threshold (Yes inS406), the decoding unit 103 decodes only the left-eye code signals 112Land thereby generates the left-eye decode signals 113L. The output unit106 outputs only the left-eye decode signals 113L as the output videosignals 117 (S407). This allows the display panel 26 to display the 2Dvideo images in the fast playback mode.

On the other hand, when the depth determining unit 305 determines thatthe amount of change in the depth is smaller than the fourth threshold(No in S406), the decoding unit 103 decodes the left-eye code signals112L and the right-eye code signals 112R and thereby generates theleft-eye decode signals 113L and the right-eye decode signals 113R. Theoutput unit 106 outputs the left-eye decode signals 113L and theright-eye decode signals 113R as the output video signals 117 (S402).This allows the display panel 26 to display the 3D video images in thefast playback mode.

When the playback mode received by the playback mode receiving unit 304is the trick play mode which is other than the reverse playback mode andthe fast playback mode (No in S401, No in S402, and No in S403), thatis, when the playback mode received by the playback mode receiving unit304 is the slow playback mode or the frame-by-frame playback mode, thedecoding unit 103 decodes the left-eye code signals 112L and theright-eye code signals 112R and thereby generates the left-eye decodesignals 113L and the right-eye decode signals 113R. The output unit 106outputs the left-eye decode signals 113L and the right-eye decodesignals 113R as the output video signals 117 (S402). This allows thedisplay panel 26 to display the 3D video images in the slow playbackmode or in the frame-by-frame playback mode.

As above, in the fast playback mode, the 3D video decoding apparatus 300according to the fourth embodiment of the present invention provides the2D presentation by outputting the left-eye decode signals 113L only.This allows the 3D video decoding apparatus 300 to prevent the 3D videoimages which intensely change in the depth, from being displayed in thefast playback mode.

Furthermore, the 3D video decoding apparatus 300 according to the fourthembodiment of the present invention provides: the 2D presentation whenthe playback speed in the fast playback mode is equal to or greater thana predetermined threshold; and the 3D presentation when the playbackspeed in the fast playback mode is smaller than the predeterminedthreshold. This allows the 3D video decoding apparatus 300 to providethe 2D presentation when the change in the depth is intense due to ahigh playback speed. In addition, the 3D video decoding apparatus 300provides the 3D presentation when the playback speed is relatively low.

Furthermore, in the fast playback mode, the 3D video decoding apparatus300 according to the fourth embodiment of the present inventionprovides: the 2D presentation when the depth intensely changes; and the3D presentation when the change in the depth is small. This allows the3D video decoding apparatus 300 to provide the 2D presentation when thechange in the depth is intense. In addition, the 3D video decodingapparatus 300 provides the 3D presentation when the change in the depthis small.

Thus, the 3D video decoding apparatus 300 according to the fourthembodiment of the present invention is capable of generating favorablevideo images in the trick play mode.

Furthermore, the 3D video decoding apparatus 300 according to the fourthembodiment of the present invention performs the 2D presentation in thereverse playback mode. In the reverse playback mode, the decodingprocess has a higher processing amount than that in the normal playbackmode. This is because a picture to be referred to by the current pictureto be decoded is predetermined on the premise that the pictures aredisplayed in the forward direction. In the reverse playback mode, ittherefore becomes necessary to decode the picture which is to bereferred to by the current picture to be decoded. This means that, inthe case of reproducing, in the reverse playback mode, the picture whichis decoded last in a sequence (GOP) in the forward playback mode, allthe reference coded pictures (I pictures and P pictures) within thesequence need to be decoded. It thus takes a longer time to decode apicture closer to the end of a sequence. Thus, when the processingcapability of the decoding unit 103 is not sufficient, there are unequaldisplay intervals; that is, pictures closer to the end of a sequencehave a longer display interval, in the reverse playback mode.Alternatively, when all the pictures have the display interval which isequal to the display interval for the most-time-consuming decodingprocess, there arises a problem of a decrease in the playback speed inthe reverse playback mode.

On the other hand, the 3D video decoding apparatus 300 according to thefourth embodiment of the present invention provides the 2D presentationin the reverse playback mode so that the processing amount of thedecoding unit 103 can be lower as compared to the case of the 3Dpresentation. This allows the 3D video decoding apparatus 300 to preventthe above-mentioned unequal display intervals and thereby generatefavorable video images in the trick play mode. Furthermore, the 3D videodecoding apparatus 300 is capable of improving the playback speed in thereverse playback mode. In addition, the 3D video decoding apparatus 300is capable of improving the response in the reverse frame-by-frameplayback mode.

The above describes the 3D video decoding apparatus 300 according to thefourth embodiment of the invention which performs the followingprocesses: (i) a process of providing the 2D presentation in the fastplayback mode; (ii) a process of providing the 2D presentation when theplayback speed in the fast playback mode is equal to or greater than apredetermined threshold, and providing the 3D presentation when theplayback speed in the fast playback mode is smaller than thepredetermined threshold; (iii) a process of providing the 2Dpresentation when the depth intensely changes, and providing the 3Dpresentation when the change in the depth is small; and (iv) providingthe 2D presentation in the reverse playback mode, but it is sufficientthat the 3D video decoding apparatus 300 performs one or more of theabove processes.

Furthermore, the order of the processes shown in FIG. 22 is an example,and other orders of the processes are applicable as long as they canlead to the same or like effects. Part of the processes may be performedat the same time.

Furthermore, while the above describes the 3D video decoding apparatus300 which provides the 2D presentation when the playback speed in thefast playback mode is equal to or greater than a predetermined speed(Yes in S405), the 3D video decoding apparatus 300 may further select toprovide the 2D presentation or to provide the 3D presentation, accordingto whether or not the amount of change in the depth is equal to orgreater than a threshold as in Step S406.

While the above describes the 3D video decoding apparatuses 100, 200,and 300 according to the first to fourth embodiments of the presentinvention, the present invention is not limited to these embodiments.

For example, the above description illustrates an example where a pairof dedicated glasses (the shutter glasses 43) is used, but the presentinvention is applicable also to a system which uses no dedicatedglasses.

Furthermore, while the above description illustrates an example whereeach of the 3D video decoding apparatuses 100, 200, and 300 according tothe implementations of the present invention is applied to a digitaltelevision and a digital video recorder, the 3D video decodingapparatuses 100, 200, and 300 according to the implementations of thepresent invention may be applied to 3D video display devices (such asmobile phone devices and personal computers) other than the digitaltelevision, which display 3D video. Furthermore, the 3D video decodingapparatuses 100, 200, and 300 according to the implementations of thepresent invention are applicable to 3D video output devices (such as BDplayers) other than the digital video recorder, which output 3D video.

Furthermore, each of the above 3D video decoding apparatuses 100, 200,and 300 according to the first to fourth embodiments is typicallyimplemented as a large-scale integration (LSI) that is an integratedcircuit. Components may be each formed into a single chip, and it isalso possible to integrate part or all of the components in a singlechip.

This circuit integration is not limited to the LSI and may be achievedby providing a dedicated circuit or using a general-purpose processor.It is also possible to utilize a field programmable gate array (FPGA),with which LSI is programmable after manufacture, or a reconfigurableprocessor, with which connections, settings, etc., of circuit cells inLSI are reconfigurable.

Furthermore, if any other circuit integration technology to replace LSIemerges thanks to semiconductor technology development or otherderivative technology, such technology may, of course, be used tointegrate the processing units.

Moreover, the processor such as CPU may execute a program to performpart or all of the functions of the 3D video decoding apparatuses 100,200, and 300 according to the first to fourth embodiments of the presentinvention.

Furthermore, the present invention may be the above program or arecording medium on which the above program has been recorded. It goeswithout saying that the above program may be distributed via acommunication network such as the Internet.

Furthermore, the present invention may be implemented as a 3D videodecoding method which includes, as steps, characteristic means includedin the 3D video decoding apparatus. The present invention may also beimplemented as a 3D video display apparatus, such as a digitaltelevision, which includes the above-described 3D video decodingapparatus, and implemented as a 3D video display system which includesthe 3D video display apparatus.

Furthermore, it may also be possible to combine at least part offunctions of the above-described 3D video decoding apparatuses 100, 200,and 300 according to the first to fourth embodiments and variationsthereof.

All the numbers herein are given as examples to provide specificexplanations of the present invention, and the present invention is thusnot restricted by those numbers.

Furthermore, the structures of the above-described 3D video decodingapparatuses 100, 200, and 300 are given as examples to provide specificexplanations of the present invention, and thus, the 3D video decodingapparatus according to an implementation of the present invention doesnot necessarily require all the above structures. In other words, it issufficient that the 3D video decoding apparatus according to animplementation of the present invention includes only the minimumstructure that can provide an effect of the present invention.

Likewise, the above-described 3D video decoding methods executed by the3D video decoding apparatuses are given as examples to provide specificexplanations of the present invention, and thus, the 3D video decodingmethod executed by the 3D video decoding apparatus, according to animplementation of the present invention, does not necessarily requireall the above steps. In other words, it is sufficient that the 3D videodecoding method according to an implementation of the present inventionincludes only the minimum steps that can provide an effect of thepresent invention. In addition, the execution order of the above stepsis given as an example to provide specific explanations of the presentinvention and therefore may be other order than that illustrated above.A part of the above step may be carried out at the same time as (inparallel with) another step.

Furthermore, the present invention encompasses various embodiments thatare obtained by making various modifications which those skilled in theart could think of, to the present embodiments, without departing fromthe spirit or scope of the present invention.

Although only some exemplary embodiments of this invention have beendescribed in detail above, those skilled in the art will readilyappreciate that many modifications are possible in the exemplaryembodiments without materially departing from the novel teachings andadvantages of this invention. Accordingly, all such modifications areintended to be included within the scope of this invention.

INDUSTRIAL APPLICABILITY

The present invention is applicable to 3D video decoding apparatuses andparticularly to digital video recorders, digital televisions, and thelike.

What is claimed is:
 1. A three-dimensional (3D) video decoding apparatuswhich decodes a first code signal obtained by coding a video signal of afirst view, and a second code signal obtained by coding a video signalof a second view that is different from the first view, said 3D videodecoding apparatus comprising: a decoding unit configured to (i) decodethe first code signal with reference to a previously decoded firstdecode signal that has already been decoded, to generate a first decodesignal, and (ii) decode the second code signal with reference to thepreviously decoded first signal and a previously decoded second decodesignal that has already been decoded, to generate a second decodesignal; an error determining unit configured to determine whether or notthere is an error in the first code signal and in the second codesignal; an output determining unit configured to determine, when saiderror determining unit determines that there is an error in one of thefirst and the second code signals assigned with correspondingpresentation time points and that there is no error in the other of thefirst and the second code signals, whether the one of the first and thesecond code signals is the first code signal or the second code signal;and an output unit configured not to output the first or the seconddecode signal that corresponds to the one or the other of the first andthe second code signals, when said output determining unit determinesthat the one of the first and the second code signals is the first codesignal, and to output only the first decode signal which is obtained bydecoding the other of the first and the second code signals, when saidoutput determining unit determines that the one of the first and thesecond code signals is the second code signal.
 2. The 3D video decodingapparatus according to claim 1, wherein the second code signal includesreference area designating information that is associated with a codedpicture included in the second code signal and designates, as areference area, a part of a decoded picture included in the first decodesignal, said decoding unit is configured to decode the coded pictureincluded in the second code signal, with reference to the reference areaincluded in the decoded picture assigned with the presentation timepoint corresponding to the coded picture, said error determining unit isconfigured to determine, for each of a plurality of slices included inthe coded picture, whether or not there is an error in the slice, saidoutput determining unit is configured to determine, when said outputdetermining unit determines that the one of the first and the secondcode signals is the second code signal, whether or not an error slicedetermined as including an error is included in the reference area, andsaid output unit is configured not to output the first or the seconddecode signal that corresponds to the one or the other of the first andthe second code signals, when said output determining unit determinesthat the error slice is included in the reference area, and to outputonly the first decode signal which is obtained by decoding the other ofthe first and the second code signals, when said output determining unitdetermines that the error slice is not included in the reference area.3. A three-dimensional (3D) video decoding method of decoding a firstcode signal obtained by coding a video signal of a first view, and asecond code signal obtained by coding a video signal of a second viewthat is different from the first view, said 3D video decoding methodcomprising: decoding the first code signal with reference to apreviously decoded first decode signal that has already been decoded, togenerate a first decode signal, and decoding the second code signal withreference to the previously decoded first signal and a previouslydecoded second decode signal that has already been decoded, to generatea second decode signal; determining whether or not there is an error inthe first code signal and in the second code signal; determining, whenit is determined in said determining of an error that there is an errorin one of the first and the second code signals assigned withcorresponding presentation time points and that there is no error in theother of the first and the second code signals, whether the one of thefirst and the second code signals is the first code signal or the secondcode signal; and not outputting the first or the second decode signalthat corresponds to the one or the other of the first and the secondcode signals, when it is determined in said determining of the one ofthe first and the second code signals that the one of the first and thesecond code signals is the first code signal, and outputting only thefirst decode signal which is obtained by decoding the other of the firstand the second code signals, when it is determined in said determiningof the one of the first and the second code signals that the one of thefirst and the second code signals is the second code signal.